14 research outputs found
Recommended from our members
Reinforcement Learning for Generative Art
Reinforcement learning (RL) is an efficient class of sequential decision-making algorithms that have achieved remarkable success in a broad range of applications, such as robotic manipulations, strategic games, or autonomous driving. The most well-known example of reinforcement learning is AlphaGo, a computer program that plays the board game Go and outperforms top human Go players. Unlike other two major machine learning categories, supervised learning and unsupervised learning, in which media artists are actively engaged, reinforcement learning has yet to result in many creative applications. Generative art is usually driven, in whole or in part, by autonomous systems that are derived from a set of rules. Interestingly, an RL policy can be seen as an autonomous system where the rules are learned by interacting with its environment. Regardless of its initial purpose, reinforcement learning has the potential to expand the boundary of generative art. However, a formal process of applying reinforcement learning to generative art does not yet exist and the current RL tools require an in-depth understanding of RL concepts. To bridge the gap, the first part of the dissertation introduces a conceptual framework to adapt reinforcement learning for generative art. The framework proposes a term RL-based generative art to denote a novel form of generative art of which the use of RL agents is the key element. The creative process of RL-based generative art and possible emergent behaviors are discussed in the framework. This leads to a discussion of several author's related practices on generative art, deep-learning art, and reinforcement learning. Those practices are critical for understanding the conceptual and technical details of each component in order to construct the framework. The second part introduces RL5, a JavaScript library for rapidly prototyping RL environments and training RL policies in web browsers. The library combines RL algorithms and RL environments into one framework and is fully compatible with p5.js. RL5 is developed with a particular focus on simplicity to favor (re)usability of RL algorithms and development of RL environments. Specifically, the library implemented three RL algorithms, Tabular Q-learning, REINFORCE, and DDPG, to cover all the three families of model-free RL, and nine RL environments that six of them address autonomous agents in steering behaviors, which can be used as building blocks for complex systems. Finally, the author demonstrates four different use cases of how to apply RL5 for pedagogical and creative applications
Representation Learning for Sequential Volumetric Design Tasks
Volumetric design, also called massing design, is the first and critical step
in professional building design which is sequential in nature. As the
volumetric design process is complex, the underlying sequential design process
encodes valuable information for designers. Many efforts have been made to
automatically generate reasonable volumetric designs, but the quality of the
generated design solutions varies, and evaluating a design solution requires
either a prohibitively comprehensive set of metrics or expensive human
expertise. While previous approaches focused on learning only the final design
instead of sequential design tasks, we propose to encode the design knowledge
from a collection of expert or high-performing design sequences and extract
useful representations using transformer-based models. Later we propose to
utilize the learned representations for crucial downstream applications such as
design preference evaluation and procedural design generation. We develop the
preference model by estimating the density of the learned representations
whereas we train an autoregressive transformer model for sequential design
generation. We demonstrate our ideas by leveraging a novel dataset of thousands
of sequential volumetric designs. Our preference model can compare two
arbitrarily given design sequences and is almost 90% accurate in evaluation
against random design sequences. Our autoregressive model is also capable of
autocompleting a volumetric design sequence from a partial design sequence
Assemble Them All: Physics-Based Planning for Generalizable Assembly by Disassembly
Assembly planning is the core of automating product assembly, maintenance,
and recycling for modern industrial manufacturing. Despite its importance and
long history of research, planning for mechanical assemblies when given the
final assembled state remains a challenging problem. This is due to the
complexity of dealing with arbitrary 3D shapes and the highly constrained
motion required for real-world assemblies. In this work, we propose a novel
method to efficiently plan physically plausible assembly motion and sequences
for real-world assemblies. Our method leverages the assembly-by-disassembly
principle and physics-based simulation to efficiently explore a reduced search
space. To evaluate the generality of our method, we define a large-scale
dataset consisting of thousands of physically valid industrial assemblies with
a variety of assembly motions required. Our experiments on this new benchmark
demonstrate we achieve a state-of-the-art success rate and the highest
computational efficiency compared to other baseline algorithms. Our method also
generalizes to rotational assemblies (e.g., screws and puzzles) and solves
80-part assemblies within several minutes.Comment: Accepted by SIGGRAPH Asia 2022. Project website:
http://assembly.csail.mit.edu
Recommended from our members
Reinforcement Learning for Generative Art
Reinforcement learning (RL) is an efficient class of sequential decision-making algorithms that have achieved remarkable success in a broad range of applications, such as robotic manipulations, strategic games, or autonomous driving. The most well-known example of reinforcement learning is AlphaGo, a computer program that plays the board game Go and outperforms top human Go players. Unlike other two major machine learning categories, supervised learning and unsupervised learning, in which media artists are actively engaged, reinforcement learning has yet to result in many creative applications. Generative art is usually driven, in whole or in part, by autonomous systems that are derived from a set of rules. Interestingly, an RL policy can be seen as an autonomous system where the rules are learned by interacting with its environment. Regardless of its initial purpose, reinforcement learning has the potential to expand the boundary of generative art. However, a formal process of applying reinforcement learning to generative art does not yet exist and the current RL tools require an in-depth understanding of RL concepts. To bridge the gap, the first part of the dissertation introduces a conceptual framework to adapt reinforcement learning for generative art. The framework proposes a term RL-based generative art to denote a novel form of generative art of which the use of RL agents is the key element. The creative process of RL-based generative art and possible emergent behaviors are discussed in the framework. This leads to a discussion of several author's related practices on generative art, deep-learning art, and reinforcement learning. Those practices are critical for understanding the conceptual and technical details of each component in order to construct the framework. The second part introduces RL5, a JavaScript library for rapidly prototyping RL environments and training RL policies in web browsers. The library combines RL algorithms and RL environments into one framework and is fully compatible with p5.js. RL5 is developed with a particular focus on simplicity to favor (re)usability of RL algorithms and development of RL environments. Specifically, the library implemented three RL algorithms, Tabular Q-learning, REINFORCE, and DDPG, to cover all the three families of model-free RL, and nine RL environments that six of them address autonomous agents in steering behaviors, which can be used as building blocks for complex systems. Finally, the author demonstrates four different use cases of how to apply RL5 for pedagogical and creative applications
Learning Dense Reward with Temporal Variant Self-Supervision
Rewards play an essential role in reinforcement learning. In contrast to
rule-based game environments with well-defined reward functions, complex
real-world robotic applications, such as contact-rich manipulation, lack
explicit and informative descriptions that can directly be used as a reward.
Previous effort has shown that it is possible to algorithmically extract dense
rewards directly from multimodal observations. In this paper, we aim to extend
this effort by proposing a more efficient and robust way of sampling and
learning. In particular, our sampling approach utilizes temporal variance to
simulate the fluctuating state and action distribution of a manipulation task.
We then proposed a network architecture for self-supervised learning to better
incorporate temporal information in latent representations. We tested our
approach in two experimental setups, namely joint-assembly and door-opening.
Preliminary results show that our approach is effective and efficient in
learning dense rewards, and the learned rewards lead to faster convergence than
baselines.Comment: 4 pages, 6 figures, accepted to ICRA 2022 RL for Contact-Rich
Manipulation Worksho